Construction and external validation of a scoring prediction model for mortality risk within 30 days of community-acquired pneumonia in children admitted to the pediatric intensive care unit: A multicenter retrospective case-control study

In this study, we constructed and validated a scoring prediction model to identify children admitted to the pediatric intensive care unit (PICU) with community-acquired pneumonia (CAP) at risk for early death. Children with CAP who were admitted to the PICU were included in the training set and divided into death and survival groups according to whether they died within 30 days of admission. For univariate and multifactorial analyses, demographic characteristics, vital signs at admission, and laboratory test results were collected separately from the 2 groups, and independent risk factors were derived to construct a scoring prediction model. The ability of the scoring model to predict CAP-related death was validated by including children with CAP hospitalized at 3 other centers during the same period in the external validation set. Overall, the training and validation sets included 296 and 170 children, respectively. Univariate and multifactorial analyses revealed that procalcitonin (PCT), lactate dehydrogenase (LDH), activated partial thromboplastin time (APTT), and fibrinogen (Fib) were independent risk factors. The constructed scoring prediction model scored 2 points each for PCT ≥ 0.375 ng/mL, LDH ≥ 490 U/L, and APTT ≥ 31.8 s and 1 point for Fib ≤ 1.78 g/L, with a total model score of 0–7 points. When the score was ≥ 5 points, the sensitivity and specificity of mortality diagnosis in children with CAP were 72.7% and 87.5%, respectively. In the external validation set, the sensitivity, specificity, and accuracy of the scoring model for predicting the risk of CAP-related death were 64.0%, 92.4%, and 88.2%, respectively. Constructing a scoring prediction model is worth promoting and can aid pediatricians in simply and rapidly evaluating the risk of death in children with CAP, particularly those with complex conditions.


Introduction
Community-acquired pneumonia (CAP) is a leading cause of hospitalization and mortality in children aged < 5 years. [1,2]he incidence of CAP among children has substantially declined by approximately 25% as a result of reduction in risk factors for pneumonia and widespread use of pneumonia vaccines over the past decade; nonetheless, CAP remains a major cause of death worldwide. [3]The burden of CAP considerably varies across countries and regions, with 95% of new CAP cases in children aged < 5 years occurring globally each year in developing countries.Of these children, 15% require hospitalization, [4,5] whereas some hospitalized children must be admitted to the pediatric intensive care unit (PICU) for treatment. [1][8] Consequently, the morbidity and mortality rates of children admitted to the PICU remain high in developing countries owing to differences among regions and other practical measures. [9,10]Accurate identification of children with CAP who are at risk for death, especially in developing countries and poor regions, has enormous implications for the subsequent reduction of child mortality.
Current predictions for childhood pneumonia focus on grading the severity of pneumonia, [11,12] and some previous studies have identified risk factors for CAP-related death in the PICU, [13,14] thereby providing a reference for identifying severe pneumonia by pediatricians.Nevertheless, pneumonia progresses rapidly in children, and the clinical presentation of children with CAP varies widely according to severity. [15]urthermore, there currently exists a controversy concerning the grading of pneumonia severity for assessment. [16]The commonly accepted prediction score for mortality in children is the Pediatric Index of Mortality 3; however, its calculation is complex and not easily generalizable. [17]When dealing with actual patients, direct identification of children at risk for death using simple, easy-to-use, objective-scoring prediction model markers may be more favorable for subsequent evaluation and treatment.To date, no study has yet reported on a prediction model for the risk of CAP-related mortality in the PICU.Therefore, in the present study, we constructed and validated a scoring prediction model for CAP-related mortality risk in the PICU to help pediatricians in condition assessment.

Participants
For the training set, 296 children admitted to the PICU of Wuhan Children's Hospital from January 2019 to July 2022 who met the inclusion criteria were included in the training set.Among these children, 263 children with CAP (163 males and 100 females; median age: 10.0 [3.0, 26.0] months) survived within 30 days of admission, whereas 33 children with CAP (24 males and 9 females; median age: 13.0 [4.5, 50.5] months) died within 30 days of admission.
For the external validation set, 170 eligible children with CAP who were hospitalized at Tongcheng People's Hospital, Macheng People's Hospital, and Jingmen Second People's Hospital during the same period were included.Among them, 145 children with CAP (86 males and 59 females; median age: 60.0 [21.0, 98.0] months) survived within 30 days of admission, whereas 25 children with CAP (17 males and 8 females; median age: 45.0 [7.5, 101.5] months) died within 30 days of admission.

Diagnostic criteria for CAP
CAP was defined as a pneumonia in a previously healthy child who acquired the infection outside the hospital.This included evidence of acute infection, such as fever, hypothermia, leukocytosis, or leukopenia; evidence of acute respiratory disease, such as new cough or sputum, chest pain, dyspnea, shortness of breath, abnormal pulmonary examination findings, or respiratory failure; and evidence consistent with pneumonia, as assessed using chest radiography or computed tomography prior to admission or within 72 hours of admission.Radiographic evidence of pneumonia included the presence of solid lesions (dense or fluffy opacities with or without signs of bronchial inflation), other infiltrates (linear and patchy alveolar or interstitial densities), and pleural effusion. [18]he inclusion criteria were as follows: children aged between 1 month and 18 years who met the diagnostic criteria for CAP and completed chest radiography or lung computed tomography within 72 hours before and after admission.The exclusion criteria were as follows: children who developed fever or respiratory illness within 14 days prior to admission, had a final diagnosis of immunodeficiency or tuberculosis, lacked data, or abandoned or interrupted the treatment.

Data collection
Data of the study participants were obtained from an inpatient electronic medical record system (Kaihua, Beijing, China).Medical records must provide detailed basic and demographic data of all children, including the hospitalization date, discharge date, discharge outcome, hospitalization number, sex, age, height, weight, and body mass index.Additionally, vital signs at admission, such as the respiratory rate, heart rate, systolic blood pressure, diastolic blood pressure, and oxygen saturation (SpO 2 ), were recorded.Laboratory tests were completed on the day of admission and included random blood glucose, blood gas analysis, routine blood examination, biochemical parameters (liver function, kidney function, cardiac enzymes, and electrolytes), coagulation, and infection parameters (highsensitivity C-reactive protein [hsCRP] and procalcitonin [PCT]).Information obtained from the medical records was recorded by a dedicated professional and carefully proofread by another professional.
This study was approved by the Ethics Committee of Wuhan Children's Hospital (approval no.: 2022R077).The requirement for the acquisition of informed consent from the participants was waived owing to the retrospective nature of this study.

Treatment protocol
Immediately after the initial diagnosis of CAP was established, the standard treatment for pneumonia (i.e., anti-infective treatment plus supportive therapy) was administered. [18]

Laboratory tests
All blood test results of the children were completed on the same day of admission, and the collected blood specimens were sent to the laboratory department for on-machine testing within 2 hours.The test results were checked by a dedicated person in the laboratory department and uploaded to the electronic medical record system.Blood and biochemical tests were regularly calibrated and tested for precision.The coefficient of variation was at the standard limit, and the instruments were controlled daily using high-and low-concentration quality control products.

Statistical analysis
Statistical analysis was performed using SPSS version 23.0 (IBM Corp., New York, NY), with statistical significance being set at P < .05.In the training set, the children were first divided into 2 groups (namely, the death and survival groups) according to the presence or absence of death within 30 days of admission.The normal distribution of continuous variables in the 2 groups was assessed using the Shapiro-Wilk test.Comparison of normally distributed variables, expressed as x ± s, was conducted between the 2 groups using a t test.Differences in non-normally distributed variables, described as median (25th, 75th), were determined between the groups using the Mann-Whitney U test.Dichotomous variables were compared between the groups using the chi-squared test and expressed as four-cell tables and composition ratios.Variables showing a significant difference (P < .05) between the groups in the univariate analysis were included in the multivariate binary logistic regression analysis.The correlation and strength between the predictor variables in the regression model were measured using the variance inflation factor, in which variance inflation factor < 10 indicated "free of covariance."Prior to binary logistic regression, the maximum Youden index for continuous variables was calculated using receiver operating characteristic (ROC) curves to determine the cutoff values and subsequently converted to dichotomous variables based on these cutoff values.Variables that were statistically different in the univariate analysis were included in the binary logistic regression equation, and independent risk factors were derived using the backward elimination method.The model was evaluated using the ROC curve, calibration plot (R version 4.3.2,calibrate, MASS package), and Hosmer-Lemeshow test.In order to increase the usefulness of risk stratification in clinical practice, we approximated it using a simple scoring model, in which integer score points were assigned to each variable by rounding the odds ratios (ORs).The total assignment score of each child was calculated.ROC curves were employed to calculate the cutoff score for the entire scoring model and to determine the sensitivity and specificity for predicting the risk of death in children.Finally, the sensitivity, specificity, and accuracy of the scoring model in diagnosing CAP-related death were examined using external validation.

Demographic characteristics and vital signs on admission in the training set
In the training set, the survival and death groups of children with CAP exhibited no significant differences with respect to sex, age, height, weight, body mass index, systolic blood pressure, diastolic blood pressure, respiratory rate, heart rate, and SpO 2 (P > .05;Table 1).

Laboratory tests at admission for children with CAP in
the training set 3.2.1.Random blood glucose and blood gas analysis.Blood glucose and blood gas analysis indicators included the following 7 main variables: blood glucose, pH, oxygen pressure (PO 2 ), carbon dioxide pressure (PCO 2 ), HCO 3 (−), actual bicarbonate (ABE), and lactic acid.HCO 3 (-) and ABE were statistically different between the death and survival groups with CAP (P < .05); in particular, both HCO 3 (−) (20.8 vs 22.9 mmol/L, P = .024)and ABE (−3.4 vs −1.6, P = .014)were lower in the death group than in the survival group (Table 2).

Conversion of continuous variables to dichotomous variables.
Seventeen continuous variables (namely, HCO 3 (−), ABE, LYM, Hb, PLT, hsCRP, PCT, AST, TP, ALB, TB, LDH, SCr, BUN, PT, APTT, and Fib) in the univariate analysis significantly differed between the survival and death groups (P < .05).ROC curves were first utilized for clinical use to calculate the area under the curve (AUC), P value, and Youden index of each continuous variable for diagnosing CAP-related death.The cutoff values, sensitivity, and specificity corresponding to the maximum Youden index were determined.Subsequently, continuous variables were transformed into dichotomous variables according to the cutoff values (Table 3).A scoring model was constructed to facilitate its clinical application by approximating the proportion of OR values.We assigned 2 points for PCT ≥ 0.375 ng/mL and 0 point for PCT < 0.375 ng/mL; 2 points for LDH ≥ 490 U/L, and 0 point for LDH < 490 U/L; 2 points for APTT ≥ 31.8 s, and 0 point for APTT < 31.8 s; and 1 point for Fib ≤ 1.78 g/L and 0 point for Fib > 1.78 g/L.The scoring prediction model contained the 4 aforementioned indicators, and the total score was calculated for each child, with each patient receiving a score of 0 to 7 points.Subsequently, for the total score, ROC curves were used to calculate the maximum Youden index corresponding to a cutoff score of 5 points, with an AUC (95% CI) of 0.862 (0.792, 0.932) (P ≤ .001).When the model score was ≥ 5 points, the sensitivity and specificity for predicting CAP-related death were 72.7% and 87.5%, respectively (Table 4; Fig. 1A).The AUC (95% CI) of the predicted probability was 0.862 (0.792, 0.933) (P < .001),which was similar to the AUC of the scoring model (Fig. 1B).

Evaluation of the prediction model. The Hosmer-
Lemeshow test showed that the predicted and observed values were not substantially different (P > .05),indicating good model suitability.Figure 1C presents the calibration curve.The total score of all patients who died or survived within 30 days in the external validation set was calculated to validate the efficiency of the scoring model.According to the scoring model, patients with a total score of ≥5 or <5 were suspected of death or survival, respectively (Table 5).Based on the scoring model, the outcomes were compared with the patients' clinical outcomes.The sensitivity, specificity, and accuracy of the scoring model in distinguishing death from survival were 64.0%, 92.4%, and 88.2%, respectively.

Discussion
PICU admission has a substantial mortality rate for CAP CAP in the PICU has a substantially high mortality rate; therefore, we developed a scoring prediction model to identify children at risk for early CAP mortality.The predictive scoring model comprised  4 independent risk factors: PCT, LDH, APTT, and Fib.We assigned 2 points each for PCT ≥ 0.375 ng/mL, LDH ≥ 490 U/L, and APTT ≥ 31.8 s, and 1 point for Fib ≤ 1.78 g/L, with a total score of 0 to 7 points.When a child's total score was ≥ 5 points, the sensitivity and specificity for predicting death in children with CAP were 72.7% and 87.5%, respectively.As validated externally in multiple centers, the sensitivity, specificity, and accuracy for predicting CAP-related death were 64.0%, 92.4%, and 88.2%, respectively.In the prediction model, the first indicator is PCT, which is produced by thyroid C cells and is barely detectable in the blood of healthy individuals but increases during severe bacterial infections, sepsis, and multiorgan dysfunction. [19]In CAP, whether PCT can accurately distinguish bacterial from viral infections controversial. [20][23] Our results indicated that the median PCT was significantly lower in the survival group than in the death group (0.25 vs 1.26 ng/mL; P < .001).With PCT ≥ 0.375 ng/mL, the sensitivity and specificity for diagnosing CAP-related death were 87.9% and 57.4%, respectively, suggesting that when the PCT index was applied singly, the sensitivity was high, but the specificity was low.Some surviving children were misclassified as dead children.The specificity was improved by constructing a model instead of a single indicator; in the scoring model, PCT ≥ 0.375 ng/mL was scored as 2 points.
The second indicator is LDH, a cytoplasmic enzyme found in all major organ systems [24] that is released into the bloodstream during tissue damage, resulting in an increase in serum LDH levels, which can be used to reflect cellular damage or inflammation. [25]29][30] Our results confirmed that LDH was significantly higher in the death group than in the survival group (692 vs 353 U/L; P < .001).On the regression analysis, the OR (95% CI) for the diagnosis of CAP with LDH ≥ 490 U/L was 4.809 (1.899-12.18),suggesting a 4.809-fold increased risk of death with LDH ≥ 490 U/L compared with LDH < 490 U/L.The sensitivity and specificity of LDH alone for predicting death were 75.8% and 73.0%, respectively, which achieved satisfactory results.This finding suggests that, in clinical practice, pediatricians must pay attention to LDH and be alert to the risk of exacerbation or even death with short-term elevation.LDH ≥ 490 U/L also ended up with a score of 2 points in our scoring model.
The third and fourth indicators in the prediction model were APTT and Fib (coagulation), which are common in CAP and increase with disease severity and poor prognosis. [31]Primary studies have focused on assessing the effects of D2 polymers on the risk of death due to pneumonia. [32]Abnormal coagulation in CAP may be caused by coagulation disorders and inflammation, which are typical host responses to infections in patients with severe sepsis. [33]At the same time, CAP is an important cause of severe sepsis. [34]Our study showed that all coagulation indices in children with CAP who died significantly differed and that PT and APTT were higher in the death group than in the survival group.Additionally, Fib was lower in the death group than in the survival group, indicating that APTT and Fib were independent risk factors for death.The OR (95% CI) for the diagnosis of CAP-related death with APTT ≥ 31.8 s was 5.752 (1.837-18.009),and the sensitivity and specificity for predicting death were 87.9% and 48.7%, respectively.The OR (95% CI) for the diagnosis of CAP-related death with Fib ≤ 1.78 g/L was 2.779 (1.101-7.012),and the sensitivity and specificity for predicting death were 45.5% and 81.4%, respectively.These results suggest that APTT and Fib alone had low specificity or sensitivity for prediction.When these 2 indicators were included in the scoring model, they were scored 2 and 1, respectively.All of the 4 aforementioned clinical indicators (namely, PCT, LDH, APTT, and Fib) correlated with CAP severity and death.A combined prediction with high sensitivity and specificity could be achieved when an OR value was assigned.Ultimately, the sensitivity and specificity of our prediction model for determining CAP-related death in children were 72.7% and 87.5%, respectively, exceeding those of a single predictor.When encountering a complex patient, calculating the relevant score first can be performed as a basis for condition assessment and decision-making.
The present study has some limitations.First, this was a retrospective study involving inpatients with treatment protocols primarily based on diseases, without subgroup interventions.As the PICU admits the most severe CAP in the region, some bias exists, and a multicenter prospective study needs to be refined.Second, the prediction model might require additional sensitivity indicators for refinement to improve its sensitivity.Nevertheless, the significance of our study is that the mortality rate of CAP in the PICU was high, and the early identification of    children at risk for death may provide a more substantial basis for physician communication with families, decision-making, and treatment.

Conclusion
CAP in the PICU has a high mortality rate, and the early identification of children with CAP at risk for death is challenging for PICU physicians.Previous pediatric mortality scores have been complex and controversial.We identified independent risk factors for death due to CAP via a retrospective study and constructed a scoring prediction model that can enable better decision-making among PICU physicians when treating children with CAP.

Figure 1 .
Figure 1.Evaluation of the prediction model.(A) ROC curve of the scoring model.The AUC (95% CI) for predicting CAP-related death was 0.862 (0.789, 0.932) (P < .001).(B) ROC curve of the single indicator.The AUC (95% CI) of the predicted probability was 0.862 (0.792, 0.933) (P < .001),which was similar to the AUC of the scoring model.The AUC of the scoring model was significantly more significant than the AUC of individual variables, suggesting that the scoring model had a better predictive value than a single indicator.(C) Calibration curve.The diagonal line indicates the ideal reference line where the predicted (x-axis) and observed probability (y-axis) coincide.The closer the bias-corrected line is to the diagonal line, the more accurately the model predicts.AUC = area under the curve, CAP = community-acquired pneumonia, CI = confidence interval, ROC = receiver operating characteristic.

Table 1
Demographic characteristics and admission vital signs in the training set.
the scoring model.The transformed dichotomous variables were included in the binary logistic regression equation, with PCT, LDH, APTT, and Fib being identified as independent risk factors.

Table 2
Laboratory tests on the day of admission of the children with CAP in the training set.

Table 3
Conversion of continuous variables to dichotomous variables in the training set.

Table 4
Finding independent risk factors and constructing scoring models in the training set.

Table 5
Predictive values of scoring model in the external validation set.